Estimating Renyi Entropy of Discrete Distributions
نویسندگان
چکیده
منابع مشابه
Source coding with escort distributions and Renyi entropy bounds
Article history: Received 20 May 2009 Received in revised form 2 July 2009 Accepted 7 July 2009 Available online 15 July 2009 Communicated by A.R. Bishop PACS: 02.50.-r 05.90.+m 89.70.+c
متن کاملRenyi Entropy Estimation Revisited
We revisit the problem of estimating entropy of discrete distributions from independent samples, studied recently by Acharya, Orlitsky, Suresh and Tyagi (SODA 2015), improving their upper and lower bounds on the necessary sample size n. For estimating Renyi entropy of order α, up to constant accuracy and error probability, we show the following Upper bounds n = O(1) · 2(1− 1 α )Hα for integer α...
متن کاملShannon Entropy , Renyi Entropy , and Information
This memo contains proofs that the Shannon entropy is the limiting case of both the Renyi entropy and the Tsallis entropy, or information. These results are also confirmed experimentally. We conclude with some general observations on the utility of entropy measures. A brief summary of the origins of the concept of physical entropy are provided in an appendix.
متن کاملBayesian entropy estimation for countable discrete distributions
We consider the problem of estimating Shannon’s entropy H from discrete data, in cases where the number of possible symbols is unknown or even countably infinite. The Pitman-Yor process, a generalization of Dirichlet process, provides a tractable prior distribution over the space of countably infinite discrete distributions, and has found major applications in Bayesian non-parametric statistics...
متن کاملEstimating Multivariate Discrete Distributions Using Bernstein Copulas
Measuring the dependence between random variables is one of the most fundamental problems in statistics, and therefore, determining the joint distribution of the relevant variables is crucial. Copulas have recently become an important tool for properly inferring the joint distribution of the variables of interest. Although many studies have addressed the case of continuous variables, few studie...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Information Theory
سال: 2017
ISSN: 0018-9448,1557-9654
DOI: 10.1109/tit.2016.2620435